An example of a conventional imaging device having a flicker reduction function is disclosed in Patent Literature 1. According to this example, integration is performed on pixel values for each horizontal line, and line integrals resulting from the integration are stored such that the number of the stored line integrals is equal to the number of frames, which is equivalent to the greatest common divisor of a flicker cycle and a frame cycle. Calculation is performed to obtain an average of the line integrals relating to the horizontal lines at the same vertical position in the respective frames. The obtained average is output as a line integral from which the flicker has been reduced. Analysis on flicker component is performed based on the line integral normalized by the average.
FIG. 8 is a block diagram showing a principal structure of a commonly-used conventional imaging device having a flicker reduction function such as the imaging device disclosed in the Patent Literature 1.
According to an imaging device 1000 shown in FIG. 8, an imaging optical component 1001 is composed of a lens, a diaphragm, and so on. An imaging element 1002 is an area image sensor that converts light, which is formed as an image on an imaging surface, into an electrical signal. The imaging optical component 1001 adjusts an amount of light from a subject by the diaphragm, and forms an image of the subject on the imaging surface of the imaging element 1002.
A system controller 1003 is composed of a microprocessor and so on, and controls the whole imaging device 1000. An optical driver 1004 adjusts a zoom, a focal point, the diaphragm, and so on of the imaging optical component 1001, in accordance with an instruction issued by the system controller 1003. An IS control unit (image sensor control unit) 1005 controls the imaging element 1002 in accordance with an instruction issued by the system controller 1003.
An AFE (analog front end) unit 1006 performs, under control of the system controller 1003, analog signal processing on the analog image signal output by the imaging element 1002, and outputs a digital image signal resulting from the analog signal processing. A digital signal processing unit 1007 performs digital signal processing on the digital image digital output by the AFE unit 1006 to generate an image signal for output from the imaging device 1000. The digital signal processing unit 1007 is composed of a flicker reduction unit 21, a signal processing unit 1022 and so on. The flicker reduction unit 21 reduces flicker component. The signal processing unit 1022 performs, on a digital image signal, white balance adjustment, gamma adjustment, image format conversion in accordance with an output image format, and so on.
FIG. 9 is a block diagram showing a structure of the flicker reduction unit shown in FIG. 8.
According to the flicker reduction unit 1021 shown in FIG. 9, a line integration unit 1031 performs, for each horizontal line, integration on digital image signals constituting the horizontal line output by the AFE unit 1006 to obtain a line integral. A memory 1032 stores therein the line integral obtained for each horizontal line by the line integration unit 1031 such that the number of line integrals is equal to a several number of frames or fields. The description is given here based on frames.
A difference calculation unit 1033 calculates a difference between a first line integral output by the line integration unit 1031 and a second line integral read from the memory 1032, and outputs the calculated difference to a normalization unit 1035. Here, the second line integral is a line integral, which is stored in the memory 1032 and corresponds to a frame that is immediately previous to a frame of a horizontal line relating to the first line integral, and the horizontal lines are at the same position in respective frames, that is, the same vertical coordinate. An average calculation unit 1034 calculates an average of line integral based on the first line integral output by the line integration unit 1031 and the second line integrals read from the memory 1032. A normalization unit 1035 divides the difference output by the difference calculation unit 1033 by the average output by the average calculation unit 1034 to obtain a normalized difference.
A DFT (Discrete Fourier Transform) unit 1036 performs a discrete Fourier transform on the normalized difference output by the normalization unit 1035 to obtain a Fourier coefficient. A flicker coefficient calculation unit 1037 specifies an amplitude and a phase of each order of a flicker component based on the Fourier coefficient output by the DFT unit 1036. Then, based on the specified amplitudes and phases, the flicker coefficient calculation unit 1037 generates a flicker coefficient corresponding to a pixel value that is currently input to the flicker reduction unit 1021. A correction operation unit 1038 performs a flicker correction operation, which is an operation for correcting flicker on an image input to the flicker reduction unit 1021, based on the flicker coefficient output by the flicker coefficient calculation unit 1037. Specifically, the correction operation unit 1038 divides the input pixel value by a value, which is obtained by adding 1 to the flicker coefficient, to obtain a correction value from which flicker has been reduced.
The following describes the schematic operations of the flicker reduction unit 1021 of the conventional imaging device 1000 having the above structure.
The line integrals calculated by the line integration unit 1031 are stored in the memory 1032 such that the number of the stored line integrals is less than, by 1, the number of frames necessary for the average calculation unit 1034 to calculate an average. Here, the number of frames necessary for average calculation corresponds to a time period based on a frequency cycle, which is equivalent to the greatest common divisor of a flicker frequency of lighting and a frame rate, such that an average to be obtained is equal to a line integral from which an influence of flicker has been eliminated. For example, when the flicker frequency is 100 Hz and the frame rate is 60 fps (frame per second), the greatest common divisor is 20 Hz and the above time period corresponds to three frames.
The difference calculation unit 1033 calculates a difference between a line integral output by the line integration unit 1031 and a line integral read from the memory 1032 that are at the same vertical position, to obtain a difference in line integral between frames. Here, a frame relating to the line integral output by the line integration unit 1031 is immediately previous to a frame relating to the line integral read from the memory 1032. Next, the normalization unit 1035 divides the difference in line integral between the frames by an average calculated by the average calculation unit 1034 that is a line integral from which the influence of flicker has been eliminated. As a result, the difference in line integral between the frames is normalized. This results in a normalized flicker component that has a constant amplitude and has no influence of the signal strength of a captured image, thereby facilitating analysis of flicker component.
The DFT unit 1036 performs a discrete Fourier transform on the normalized difference in line integral calculated by the normalization unit 1035, with respect to the number of lines per flicker cycle. For example, when the flicker frequency is 100 Hz and the frame rate is 60 fps and the number of lines per frame is N, the number of lines per flicker cycle is N·60/100.
The calculated Fourier coefficient gives an amplitude and a phase of flicker component for respective orders. The flicker coefficient calculation unit 1037 calculates a flicker coefficient for an input pixel value based on the amplitude and the phase. Lastly, the correction operation unit 1038 performs a flicker correction operation on the pixel value input to the flicker reduction unit 1021, using the flicker coefficient output by the flicker coefficient calculation unit 1037.